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(57) Abstract 

A rhythm analyser comprises input means (2), a low-pass filter bank 
(7) including a plurality of Gaussian-type low-pass filters each having dif- 
fering cut-off frequencies, differentiating means (8) for producing a plurali- 
ty of derivative signals, detection means (9) for detecting a substantially 
zero value in at least one of the derivative signals and for producing a zero 
crossing signal, storage means (10, 11), and processing means to record 
structural information in the storage means (10, 1 1) relating to the zero 
crossing signal comprising at least one of, the position in time of the zero 
crossing signal, the identity of the low-pass filter which produced the fil- 
tered signal, or the energy of the zero crossing signal. A corresponding sys- 
tem for synthesising musical rhythmic expression comprises synthesis 
means (116) for receiving structural information and for generating synthe- 
sis signals from the structural information. 
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ANALYSIS AND SYNTHESIS OF RHYTHM 

This invention relates to a rhythm analyser for analysing 
rhythm and to a rhythmic expression synthesiser for 
5 synthesising rhythmic expression , particularly though not 
exclusively for use with signals representing a musical 
performance . 

In several fields of study it is useful to be able to 
10 consistently and accurately analyse the rhythm of a sound. 
The sound may for instance be a musical or poetic 
performance in which case the analysis yields information 
about methods of performance and the way in which the 
performance is segmented by the performer. In other fields 
15 such as the analysis of birdsong and insect chirps, the 
analysis is an aid to classification and recognition of the 
sound. In a slightly different field, the analysis of 
rhythm may be useful in analysing electro-myographic signals 
associated with muscular movements . 

20 

For example , it is known that temporal and dynamic 
variations in a performance (which variations constitute an 
expressive signal) contain significant information about the 
intended structure and interpretation provided by the 
25 performer. The structure of the music may then be 
represented as a hierarchical grouping of events such as 
notes and phrases , in a tree structure . 

A few systems exist which attempt to recover rhythmic 
30 structure from a performed sequence of music. One such 
system employs a computer program based on a grammar, which 
takes input values from a musical keyboard and groups 
musical notes according to their rhythmic structure. 
Another system uses a neural network approach to the same 
35 problem, A further system employs a signal processing 
appr ach based on a model of the auditory system and is thus 
abl to analyse music dir ctly from sound rather than from 
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electrical input: from a keyboard or other instrument. This 
latter system runs in real-time and is able to track a live 
performance . 

5 Whilst each of these systems is able to recover low-level 
rhythmic structure to varying degrees of success, they do 
not address the problem of higher level structure above the 
level of the musical bar, i.e. structures such as phrases , 
sections or even movements. Furthermore, none of these 
10 systems is able to recover fully the expressive information 
contained in a musical performance. 

Whilst systems exist for synthesising expressive 
performances, none employ a realistic representation of 
15 structure and thus considerable adjustment by hand is 
required to produce a realistic performance. 

According to a first aspect of the invention, a rhythm 
analyser comprises input means for receiving an input 
signal representing the energy of a time- varying 
electrical signal and for generating an output signal, a 
5 low-pass filter bank including a plurality of Gaussian- 
type low-pass filters having differing cut-off 
frequencies each for filtering the output signal to 
produce a plurality of respective filtered signals, 
differentiating means for receiving the filtered signals 

10 and for producing a plurality of derivative signals 
representing at least the first derivative of each 
filtered signal, detection means for detecting a 
substantially zero value in at least one of the 
derivative signals and for producing a zero crossing 

15 signal corresponding to the derivative signal having the 
substantially zero value, storage means , and processing 
means, the processing means being operable, upon receipt 
of the zero crossing signal, to record structural 
information in the storage means relating to the zero 

20 crossing signal comprising, at least one of, the position 
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in time of the part, of the input signal which corresponds 
to the zero crossing signal, the identity of the low-pass 
filter which produced rhe filtered signal which 
corresponds to the zero crossing signal, and the energy 
5 of the part of the input signal which corresponds to the 
zero crossing signal. 

According to a second aspect of the invention, a method 
of analysing rhythm comprises filtering a signal 
10 representing the energy of a time varying electrical 
signal through a low-pass filter bank comprising a 
plurality of Gaussian-type low-pass filters having 
differing cut-off frequencies to produce a plurality of 
respective filtered signals, differentiating the filtered 
15 signals to produce a plurality of derivative signals 
representing at least the first derivative of each 
filtered signal, detecting a substantially zero value in 
at least one of the derivative signals and producing a 
zero crossing signal corresponding to the derivative 
20 signal having the substantially zero value, and upon 
receipt of the zero crossing signal, recording structural 
information relating to the zero crossing signal 
comprising, at least one of , the position in time of the 
part of the input signal which corresponds to the zero 
25 crossing signal, the identity of the low-pass filter 
which produced the filtered signal which corresponds to 
the zero crossing signal, and the energy of the part of 
the input signal which corresponds to the zero crossing 
signal . 

30 

The range of time constants for the low pass filters is 
preferably between 100 seconds and 20ms. This range is 
particularly useful for the analysis of musical rhythmic 
expression. 

35 

By preferably using low-pass filters having infinite 
impulse responses, the tool may be made more useful in 



WO 93/24923 



PCT/GB93/01I83 



15 



psychological studies, since these types of filters are 
more akin to natural filter processes than those having 
finite impulse responses. 

5 Preferably, the input signal is sampled and the low-pass 
filters are digital filters. This permits much greater 
flexibility in the operating parameters of the analyser 
particularly in relation to the number, type and spacing 
of the filters in the filter bank. 

0 

Given a form of expressive representation, or having 
generated structural information, and perhaps stored it 
the information may be used to synthesise musical 
rhythiuic expression. 



According to a third aspect of the invention, a system 
for synthesising musical rhythmic expression comprises 
synthesis means for receiving structural information and 
for generating synthesis signals from the structural 

20 information, a pulse generator coupled to the synthesis 
means, for generating a pulse signal from the synthesis 
signals, and a filter means coupled to the pulse 
generator, comprising a plurality of Gaussian-type 
filters, for generating an expressive movement signal by 

25 filtering the pulse signal and summing the outputs of the 
filters . 

The expressive movement signal is preferably fed back to 

30 ^ e i+ . SyntheS / S means to b * used, in conjunction with 
30 built-m information related to typical tempo and dynamic 
variations used by a performer for particular structures , 
to provide a modulation output. The modulation output is 
preferably used to modulate the automatic performance of 
an instrument which might for example, be a MIDI-type 
35 instrument. ^ 
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Preferably the structural infestation is a library store 
of zero-crossing structures generated by the analysis 
system. It may instead be generated directly from a 
score. This score is preferably of the type used for 
5 input to a MIDI-type instrument. By combining the 
modulation output feature and the direct structural 
information generation feature, the synthesis system may 
be used to read a score and to generate a fully automated 
rhythmically expressive performance of the score with 
10 little or no manual intervention. 

According to a fourth aspect of the invention, a method 
of analysing rhythm comprises taking a signal that has 
been filtered through a Gaussian-type filter means and 
15 differentiated, in order to extract rhythmic structural 
information . 

The invention will now be described by way of example, 
with reference to the drawings in which: - 

20 

Figure 1 is a block diagram of a rhythm analyser in 
accordance with the invention; 

Figures 2(a) to 2(c) show a musical rhythmic cliche; 

25 

Figures 3(a) to 3(e) each show a zero-crossing structure 
for the cliche of Figure 2; 

Figure 4 shows an energy density spectrum (or surface) 
30 for the cliche of Figure 2; 

Figure 5 is a block diagram of a first embodiment of a 
synthesis system in accordance with the invention; and 

35 Figure 6 is a block diagram of a second embodiment of a 
synthesis system in accordance with the invention. 
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5 

A rhythm analyser in accordance with the invention may be v 
implemented in combination with a conventional computer 
with some additional hardware and software . The 
additional hardware and software requirements are 
5 described below. The computer (not shown) may for 
example, be an IBM compatible or Apple Macintosh (Trade 
Mark) type computer. The computer may have some form of 
graphical output device such as a printer or VDU, and a 
storage device such as a disk drive or tape streamer. 
10 Additionally it may have some form of user input device 
such as a keyboard and/or a pointing device such as a 
mouse . 

Alternatively, the analyser described below may be 
15 manufactured using entirely analogue components. 

With reference to Figure 1, an analogue audio signal is 
fed from an input device 1 into an input of an. interface 
card 2 installed in the computer. The input device 1 may 
20 for example, be a tape cassette, compact disk player or 
microphone . 

The card 2, comprises a full-wave rectifier and analogue 
multiplier 3, an anti-aliasing filter 4 and a 12 bit 

25 analogue-to-digital converter (ADC) 5- The rectification 
circuit used in the rectifier 3 may be an active circuit 
using two operational amplifiers and two signal diodes in 
a conventional manner. The analogue multiplier is used 
to square the audio input signal so that the output from 

30 this stage is a measure of the energy density of the 
input signal. The multiplication circuit may also 
provide any amplification and/or buffering required by 
the output of the input device 1 . 

35 The cut-off frequency of the low-pass anti-aliasing 
filter is variable to allow the analyser to analyse 
different frequencies. The cut-off frequency should be 
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at most half that of the sampling frequency of the ADC 5 
and in practice, the cut-off frequency is kept below 20Hz 
with a sampling frequency in excess of 100Hz. The filter 
4 may be a 5th-order, all-pole, maximally flat low-pass 
5 filter implemented using an LTC1062 chip, in which case 
the cut-off frequency can be adjusted by altering the 
frequency of the clock signal supplied to the chip. 

The sampling frequency of the ADC 5 is also variable and 

r 

10 should be adjusted according to the need to avoid 
aliasing of the input signal and the capability of the 
rest of the computer to accept the data output from the 
ADC 5. 

15 The output from the ADC 5 comprises a series of values 
representing the instantaneous energy density of the 
input signal, and may be processed in real time and/or 
may be stored on a storage device such as disk storage 
10, for later processing. 

20 

The main elements for processing the ADC output data are 
enclosed in box 6 in Figure 1 and comprise a digital low- 
pass filter bank 7 , differentiation means 8 and detection 
means 9. As mentioned above, although in this 

25 embodiment, the elements in box 6 are implemented in 
software, they could instead be implemented in hardware. 

The filter bank 7 comprises several Infinite Impulse 
Response (IIR) filters having a Gaussian-type response. 

30 Since a perfect Gaussian response cannot practically be 
implemented, approximations must be used and these can be 
selected from Taylor, Laguerre or Bessel series 
approximations. The order of each filter is adjustable 
and a range of between first and ninth order is 

35 acceptable in this application. The filter range is also 
adjustable i.e. the shortest and longest time constant in 
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the filter bank. Finally, the filter spacing is 
adjustable, in terms of the number of filters per octave. 

The digital implementation of the filters may be derived 
5 for example, by applying a bilinear transform to a 
suitable analogue filter design. 

The output of each filter represents the energy density 
of each frequency range or channel and therefore taking 
all channels, represents an energy density frequency 
spectrum as a function of time. The output from each 
channel is fed from the filter bank to the 
differentiating means 8. 



10 



15 The differentiating means 8 differentiates the output 
from each channel up to the nth derivative where n is 
variable. The differentiation of the filtered signal has 
the same effect as passing the filter through a band-pass 
bank with a Gaussian response and the analyser could be 

20 implemented in that way. 

The output from the differentiating means 8 is passed to 
the detection means 9 . 

A rhythmic interpretation of the input signal which in 
musical terms, can be described by metre and phrasing 
requires three kinds of information and the detection 
means is triggered under three conditions relating to 
these three types of zero crossing structure: 



25 



30 



(i) stress structure - a zero in the 1st order 
derivative in conjunction with a negative 2nd order 
derivative, in a particular channel at a particular 
time. 

35 (ii) group structure - a zero in the 1st derivative in 
conjunction with a positive second derivative in a 
particular channel at a particular time. 
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(ii) onset: structure - a negative-going zero crossing of 
the nth order derivative (typically n=2) for a 
particular channel at a particular time . 

5 The analyser is , in signal processing terms , convolving 
a Gaussian type infinite impulse response with the signal 
energy |x(t)J 2 over a range of time constants 
{x|k=0 . . .N-l} such that 

S a (t,f k ) = G{t,z k ) *| xit) | 2 = rG(t',t k ) | x(t-t') \ 2 dt / 

J o 

10 Where S(t f f ) is a windowed measure of the energy density 
spectrum, G(t,x) is a Gaussian low-pass impulse response 
function, f k = l/x k/ t is time and x is a time constant. 

In the case of onset detection, the conditions 

15 

and 

-^Sq.Ct.T*) > 0 

must be met and similar conditions as described above, 
must be met for the other two zero-crossing structures 
20 (group and stress structures). 

A musical rhythmic cliche is shown in Figure 2 and 
sample outputs from the analyser are shown in Figures 
3(a) to 3(e), for that cliche. The loci of zerr-- 
25 crossings of the filter d 2 /dt 2 G 2 (tf x k ) convolved with tiie 
signal energy from the sequence shown in Figure 2(d) is 
shown in Figure 3(a) which shows an onset structure. G 2 
denotes a second-order Gaussian approximation. The time 
constants have a range 20ms-200ms. Figures 3(b) to 3(d) 
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show stress structures. In Figure 3(b), the loci of the 
zero crossings of the filter d/dtG 3 ( t , x k ) convolved with 
the signal energy from the sequence shown in Figure 2(d) 
is shown, m Figure 3(c) the same sequence is shown with 
5 a 3dB accent on e5 and in Figure 3(d) the same sequence 
is shown again with e5 doubled in length giving the 
musical effect of legato. G 3 is a third order Gaussian 
approximation and the time constants x k have the range 
50ms-3000ms. Figure 3(e) shows a group zero crossing 
10 structure for the cliche of Figure 2. This has separated 
the groups (PI and P2 in Figure 2(a)) but for a short 
input signal such as this, it does not yield a great deal 
of information. The group structure will for example 
indicate the boundaries of a verse in poetry 

15 

When the detection means 9 is triggered, it records the 
time, channel, energy and type of trigger. This 
structural information relating to the zero crossing 
structures may be stored on the storage device 10 or may 
20 be passed on to the output processor 11. 

The storage device 10, can be used to store output data 
from the ADC 5 in the form of ASCII text files. Since 
the sampling rate is in most cases, not much more than 
100Hz, it is feasible to store long samples. ASCII text 
files may also be used for storing information relating 
to the zero-crossing structures. 

The output processor 11 is able to process three types of 
30 display which may be output on a VDD, printer or other 
type of graphical display device. 

The three types of display are: 

(i) A display of the three types of structures 12 that 
35 trigger the differentiating means. (These have been 

discuss d above in connection with Figures 2 and 3) . 



25 
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(ii) A display of the energy density spectrum 13 of the 
input signal. 

(iii) A display of analyser system parameters 14. 

5 The three structures (or zero-crossing structures) are of 
three kinds as discussed above. Each of these can be 
displayed separately or in any combination. The 
structures can be displayed as a 2D scatterplot of events 
with time on the X-axis and either time-constant or 
10 energy on the Y axis. A single event can be represented 
by a single pixel. Alternatively , the 3D aspect (i.e. 
time, time-constant and energy) of the zero crossing 
structures can be represented using colour to represent 
the energy of a particular event. In addition, any of 

15 the displays may be annotated by the operator. This 
feature may be used to label particular features of a 
zero crossing structure. 

The stress structure is used most often and resembles the 
20 tree structures used in speech and musical analysis to 
represent rhythm. 

Typically, onset structures in music for example, have 
short time constants of up to about 1 second whereas 
25 stress and group structures have longer time constants . 
For example to recover the group structure relating to 
the movements of a symphony, time constants of the order 
of the length of a movement are required i.e. 15 or 20 
minutes. Using time constants longer than the duration 
30 of the input signal will yield no additional information. 
Since the range of time constants displayed may be varied 
separately for stress ar.i group s uctures on one hand 
and for onsets on the other, the sc.- ^es of the plots may 
vary. In this case the display may be divided into 
35 separate "windows" which display simultaneously the 
global and local aspects of the structures. 
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13 

(d) digital filter bank parameters 

(i) approximation type (Bessel, Laguerre, 

Taylor, typically Taylor) 

approxi mation order (typically 6th order 
5 for stress, 2nd order for onset) 

(iii) range (typically 0.25s-30s for stress, 
0.05s-0.5s for onset) 

( iv ) filter s pacing (typically 12 per octave). 

10 The following data control parameters may be adjusted: 

(a) sample status flags, 

(*) acquire only - storage only, output 

filename requested . 
( i:L ) acquire and analyse - storage and 

15 analysis, output filename requested. 

retriev e and analyse - retrieve from 

storage and analyse, input filename 

requested . 

(b) energy density display flags, and 

20 (i) display enabled - x and y scaling factors 

requested . 

t 11 ) scroll/static - type of display, scrolled or 
repeatedly overwritten in static form. 

(c) zero-crossing structure display flags 

25 (*) define combination of display types - stress 

structure, group structure and/or onset 
structure . 



A corresponding synthesis system operates to regenerate 
a rhythmically expressive performance and has two 
separate modes:- 

(i) Reconstructive Mode in which the envelope of an 
expressive signal can be reconstructed from its 
structural analysis which forms an input. 
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(ii) Modulation mode in which the output of the synthesis 
filter can be used to modulate a performance via a 
standard MIDI interface . 

5 With reference to Figure 5 and 6 , the synthesis svstem 
comprises : - 

(a) a means of representing and retrieving a stored 
structural information 114; 



10 



15 



(b) a synthesis algorithm 116 which in Figure . 5 is the 
reverse of the analysis elements 6; 

(c) a multi-channel pulse generator 118; and 

(d) a multi-channel synthesis filter-bank 120. 



In reconstructive mode (as shown in Figure 5) the 
structural information which forms the input to the 
20 synthesis system have the same format as that generated 
by analysis system. The structures can be retrieved from 
a library. m reconstructive mode, the synthesis 
algorithm reconstructs the real-time properties of the 
structure from its compacr coding and directs to each 

25 channel a series of logic pulses of varying widths . The 
logic pulses trigger the pulse-generator which in turn 
provides impulses to the synthesis filter system which 
sxmulate the zero-crossings recorded by the analysis 
system. The synthesis filter bank mirrors the analysis 

30 filter bank used to derive the structure information. 
The parameters of the synthesis filter-bank are the same 
as those used in the analysis filter bank when the 
structural information was recorded. The response of the 
synthesis filters are summed to produce a reconstructed 

35 signal envelope. 
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15 

With reference to Figure 6, the starting point in beat 
modulation mode is a score 122 of a piece of music i.e. 
a representation of the note values necessary to drive a 
MIDI interface 124. From this the user derives a 
5 normative structure 114b i.e. one which does not have 
real-time values, but relative durations. The terminal 
nodes of this structure correspond to beats in the score 
122. 



10 The system operates from this derived structure 114b in 
much the same way as in reconstructive mode. However, in 
this mode, the expressive movement is fed back via a 
feedback path 126 to the synthesis algorithm 116b. This 
algorithm differs from that used in reconstructive mode 

15 in that it operates to modulate the tempo and dynamics of 
a modulated score 128 according to built-in "performance" 
rules which operate on the expressive movement signal. 
The modulated score 128 is output via the MIDI interface 
124 to a MIDI instrument 126. This system thereby 

20 generates an automatic expressive performance from a 
musical score. The output from the MIDI instrument is 
also controlled by style parameters such as "global 
tempo" . 
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CLAIMS 



A rhythm analyser comprising: 

input means for receiving an input signal 
representing the energy of a time-varying electrical 
signal and for generating an output signal, 

a low-pass filter bank including a plurality of 
Gaussian-type low-pass filters having differing cut- 
off frequencies each for filtering the output signal 
to produce a plurality of respective filtered 
signals, 

differentiating means for receiving the 
filtered signals and for producing a plurality of 
derivative signals representing at least the first 
derivative of each filtered signal, 

detection means for detecting a substantially 
zero value in at least one of the derivative signals 
and for producing a zero crossing signal 
corresponding to the derivative signal having the 
substantially zero value, 

storage means, and 

processing means operable, upon receipt of the 
zero crossing signal, to record structural 
information in the storage means relating to the 
zero crossing signal comprising, at least one of: 
the position in time of the part of the input signal 
which corresponds to the zero crossing signal; the 
identity of the low-pass filter which produced the 
filtered signal which corresponds to the zero 
crossing signal; and the energy of the part of the 
input signal which corresponds to the zero crossing 
signal. 

An analyser according to claim 1, wherein the low- 
pass filters have time constants in the range from 
100 seconds to 20ms. 



» 
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3. An analyser according to any preceding claim, 
wherein each low-pass filter has an infinite impulse 
response . 

5 4. An analyser according to any preceding claim, 
wherein the time-varying electrical signal is an 
audio signal . 



10 



An analyser according to any preceding claim, 
wherein the detection means is operable to detect a 
stress structure by recognising the coincidence of 
a substantially zero value in the first order 
derivative and a negative second order derivative in 
the derivative signals associated with one of the 
15 low-pass filters. 

6. An analyser according to any preceding claim, 
wherein the detection means is operable to detect a 
group structure by recognising the coincidence of a 
20 substantially zero value in the first order 

derivative and a positive second order derivative in 
the derivative signals associated with one of the 
low-pass filters. 

25 7. An analyser according to any preceding claim, 
wherein the detection means is operable to detect an 
onset structure by recognising the coincidence of a 
substantially zero value in at least the second 
order derivative and a negative value in the next 
highest order derivative in the derivative signals 
associated with one of the low-pass filters. 

8. An analyser according to claim 7, wher*in the onset 
structure is detected by recognising a substantially 
35 z ro value in the second order derivative. 



30 



An analyser according to any of claims 5 to 8, 
wherein the processing means is operable to record 
the type of structure as part of the structural 
information. 

An analyser according to any preceding claim, 
wherein the input means comprises an anti-aliasing 
filter and an ADC for sampling the input signal and 
wherein the low-pass filters are digital filters. 

An analyser according to any preceding claim, 
wherein the storage means comprises any one of a 
computer disk f a computer memory, a VDU driver 
circuit, a chart recorder, a tape streamer and a 
printer . 

A method of analysing rhythm comprising: 

filtering a signal representing the energy of 
a time varying electrical signal through a low-pass 
filter bank comprising a plurality of Gaussian-type 
low-pass filters having differing cut-off 
frequencies to produce a plurality of respective 
filtered signals, 

differentiating the filtered signals to produce 
a plurality of derivative signals representing at 
least the first derivative of each filtered signal, 

detecting a substantially zero value in at 
least one of the derivative signals and producing a 
zero crossing signal corresponding to the derivative 
signal having the substantially zero value, and 

upon receipt of the zero crossing signal, 
recording structural information relating to the 
zero crossing signal comprising, at least one of: 
the position in time of the part of the input signal 
which corr sponds to the zero crossing signal; the 
identity of the low-pass filter which produced the 
filtered signal which corresponds to the zero 
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crossing signal; and the energy of the part of the 
input signal which corresponds to *Lhe zero crossing 
signal . 



5 13. A method according to claim 12, comprising detecting 
a stress structure by recognising the coincidence of 
a substantially zero value in the first order 
derivative and a negative second order derivative in 
the derivative signals associated with one of the 
10 low-pass filters. 

14. A method according to claim 12 or to claim 13 , 
comprising detecting a group structure by 
recognising the coincidence of a substantially zero 

15 value in the first order derivative and a positive 

second order derivative in the derivative signals 
associated with one of the low-pass filters. 

15. A method according to any of claims 12 to 14 , 
20 comprising detecting an onset structure by 

recognising the coincidence of a substantially zero 
value in at least the second order derivative and a 
negative value in the next highest order derivative 
in the derivative signals associated with one of the 
25 low-pass filters . 

16. A method according to claim 15, wherein the onset 
structure is detected by recognising a substantially 
zero value in the second order derivative, 

30 

17. A method according to any of claims 12 to 16 
comprising recording the structural information on 
any one of a computer disk, a computer memory, a VDU 
driver circuit, a chart recorder, a tape streamer 

35 and a printer. 
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18. A system for synthesizing musical rhythmic 
expression comprising: 

synthesis means for receiving structural 
information and for generating synthesis signals 
5 from the structural information, 

a pulse generator coupled to the synthesis 
means, for generating a pulse signal from the 
synthesis signals, and 

a filter means coupled to the pulse generator, 
10 comprising a plurality of Gaussian-type filters, for 

generating an expressive movement signal by 
filtering the pulse signal and summing the outputs 
of the filters. 
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